As big data analytics is becoming more and more critical in the business domain, demands for big data processing platforms as a service and a multi-tenancy of the big data processing platforms are increasing. The big data processing platforms allow users to rent a large scale cluster whenever they need and pay for the service without building an expensive cluster.
In a shared cluster, many users are likely to submit jobs simultaneously and compete for resources to execute their jobs. The various jobs submitted by the users may have different time requirements and resource requirements. Thus, current job scheduling schemes are ineffective and limited in handling the different variables associated with job request from different users when submitted simultaneously or in a short time and some or the entire part of their runs are overlapped with others in a big data processing platform.